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Abstract 

This paper concerns cheaply computable formulas and bounds for the condition 
number of the TLS problem. For a TLS problem with data A, b, two formulas are 
derived that are simpler and more compact than the known results in the literature. 
One is derived by exploiting the properties of Kronecker products of matrices. The 
other is obtained by making use of the singular value decomposition (SVD) of [A b], 
which allows us to compute the condition number cheaply and accurately. We present 
lower and upper bounds for the condition number that involve the singular values of 
[A b] and the last entries of the right singular vectors of [A b] . We prove that they are 
always sharp and can estimate the condition number accurately by no more than four 
times. Furthermore, we establish a few other lower and upper bounds that involve 
only a few singular values of A and [A b]. We discuss how tight the bounds are. 
These bounds are particularly useful for large scale TLS problems since for them any 
formulas and bounds for the condition number involving all the singular values of A 
and/or [^4 b] are too costly to be computed. Numerical experiments illustrate that our 
bounds are sharper than a known approximate condition number in the literature. 

Keywords: total least squares, condition number, singular value decomposition. 
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1 Introduction 

For given A £ M™'^"(m > n), b £ M™, the TLS problem can be formulated as (see, e.g., 

min||[£; rllli;', subject to b-r en{A + E), (1) 

where || • \\f denotes the Probenius norm of a matrix and TZ{-) denotes the range space. 
Suppose tiiat [Etls rTLs] solves the above problem. Then x = xtls that satisfies the 
equation {A + Etls)x = b — ttls is called the TLS solution of ([1]). 
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The condition number measures the worst-case sensitivity of a solution of a problem 
to small perturbations in the input data. Combined with backward errors, it provides 
an approximate local linear upper bound on the computed solution. Since the 1980's, 
algebraic perturbation analysis for the TLS problem has been studied extensively; see 
O El El [l5l IH] and the references therein. In recent years, asymptotic perturbation 
analysis and TLS condition numbers have been studied; see, e.g., ^ [HI [18]. 

In the present paper, we continue our work in [8| that studied the condition number of 
the TLS problem. We will derive a number of results. Firstly, we establish two formulas 
that are simpler and more suitable for computational purpose than the known results in the 
literature. One is derived by exploiting the properties of Kronecker products of matrices. 
It improves the formulas given in j8l [18] , is independent of Kronecker products of matrices 
and makes its computation convenient. The other is obtained by making use of the SVD 
of [A b], which can be used to compute the condition number more cheaply and accurately 
than that in Secondly, we present lower and upper bounds for the condition number 
that involve the singular values of [A b] and the last entries of the right singular vectors 
of [^4 b]. We prove that these bounds are always sharp and can estimate the condition 
number accurately by no more than four times. Finally, we focus on cheaply computable 
bounds of the TLS condition number. We establish lower and upper bounds that involve 
only a few singular values of A and [A b]. We discuss how tight the bounds are. These 
bounds are particularly useful for large scale TLS problems since for them any formulas 
and bounds for the condition number involving all the singular values of A and/or [A b] 
are too costly to be computed. So we can compute these bounds by calculating only a few 
singular values of A and/or [A b] using some iterative solvers for large SVDs. In [2], an 
approximate TLS condition number is presented and is applied to evaluate conditioning of 
the TLS problem there. In this paper, we present numerical experiments to demonstrate 
a possibly great improvement of one of our upper bounds over the approximate condition 
number in [2j. 

The paper is organized as follows. In Section 2, we give some preliminaries necessary. 
In Section 3, we present computable formulas of the TLS condition number. The straight- 
forward bounds on the TLS condition number are considered in Section 4. In Section 5, 
we present numerical experiments to show the tightness of bounds for the TLS condition 
number. We end the paper with some concluding remarks in Section 6. 

Throughout the paper, for given positive integers m, n, denotes the space of n- 
dimensional real column vectors, R'"^" denotes the space of all m x n real matrices. || • || 
and II • \\f denote 2-norm and Frobenius norm of their arguments, respectively. Given 
a matrix A, A(l : i,l : i) is a Matlab notation that denotes the ith leading principal 
submatrix of A, and (Ti{A) denotes the ith largest singular value of A. For a vector a, 
a{i) denotes the ith. component of a, and diag(a) is a diagonal matrix whose diagonals are 
given as components of a. I„ denotes the n x n identity matrix, Omn denotes the m x n 
zero matrix, whereas O denotes a zero matrix whose order is clear from the context. For 
matrices A= [ai, . . . , a^,] = [Aij] G M'"^" and B, A^B = [AijB] is the Kronecker product 
of A and B, the hnear operator vec : M*"^" — ). M™" is defined by vec{A) = [aj , . . . ,a^]-^ 
for A G M'"^". 
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2 Preliminaries 



Throughout the paper, we let U^AV = diag((Ti, be the thin SVD of ^ G M"'^", 

where ai > ■ ■ ■ > a^, U e M"^^", U'^U = /„, F G M"^", V^V = In- Let U'^[A b]V = 
diag(c7i, . . . ,fTn+i) be the thin SVD of [A b] G R'^xC^+i), where ai > ■ ■ ■ > cj„+i, U = 
[ni,...,n„+i] GM'"^ C/^C/ = F = . . . , G M("+i)><("+i), F^F = In+i- 

The fohowing result presents an existence and uniqueness condition for the TLS solu- 
tion [5]. 



Theorem 1 // 

then the TLS problem (OP has a unique solution xtls- Moreover, 

(A^A-al^,I)-'A''b 



XTLS 



Vn+lin) 



Vn+i{n + 1)' ■ ■ ■ ' Vn+l{n + 1) 



(2) 

(3) 
(4) 



In the paper, it is always assumed that condition ([2]) holds. We note that, for a given 
TLS problem ([1]), if dn+i = 0, then b G Tl{A). In this case, the system of equations Ax = b 
is compatible, and we can take [E r] = O. As in [51 [TB], in the sequel, we do not consider 
this trivial case and assume that 



< (Tn+l < an- 

We will use the following properties of the TLS problem, which are in [5j: 

2 Ikf 



1 + X 



and 



A'^r 



1 + ||x||^ 

where x = xtls-, ^ = -^x — b. By (j4]), it holds that 

1 



Vn+l 



\x\ 



X 

-1 



(5) 

(6) 
(7) 

(8) 



up to a sign ±1. 

The following basic properties of the Kronecker products of matrices are needed later 
and can be found in |6]: 

{A ®C){B®D) = (AB) ^ (CD), 
{A Bf = A^(^ B^, 



where A, B, C, D are matrices with appropriate sizes. 
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3 Computable formulas for the TLS condition number 



Throughout the paper, we follow the definition of condition number in [4^ il3j. Let g : 
— > be a continuous map in normed linear spaces defined on an open set Dg C W. 
For a given a G Dg, a 7^ 0, with g(a) ^ 0, ii g is differentiable at a, then the relative 
condition number of g at a is 

^ \\9ia)\\ ^ ^ 

and the absolute condition number of g is 

Kg{a) = \\g'{a)\l (10) 

where g'{a) denotes the derivative of 5 at o. Given the TLS problem ([T]), let A = A + AA, 
b = b + Ab, where and Ab denote the perturbations in A and b, respectively. Consider 
the perturbed TLS problem 

min||[^r]||F subject to 6 - r G 7^(i + ^). (11) 

In [8], we have established the following result. 

Theorem 2 Suppose the TLS problem satisfies f^. Denote by x = xtls the TLS 
solution, and define r = Ax — b, G{x) = [x^ — 1] (S" Im- If II [AA AbJUj? is small enough, 
then the perturbed problem has a unique TLS solution xtls- Moreover, 



XTLS = XTLS + K 

where 



vec(AA) 
Ab 



0(||[AAA6]||^), (12) 



K = {A^A - al+Jn) ' (2^^^^G(x) - A^G{x) - [/„ r^ O]^ . (13) 

Denote a = \ec{A). Based on Theorem [2l in a small neighborhood of G 
|m(n+i)^ we can define the function 



g : R™("+1) 



a 
b 



i = {A'A-di^Jn)-^An, 



where a = a + vec(AA) = vec(j4), b = b + Ab, and x is the solution to the perturbed TLS 
problem (jlip . In particular, ^(([a"^, 6^]"^) = x. Thus, we have the following theorem. 

Theorem 3 Let Hg{A,b) and K'^g{A,b) be the absolute and relative condition numbers of 
the TLS problem, respectively. Then 

Kg{A,b) = \\K\\, K^g{A,b) = "^"y^^^^"^ , (14) 

where K is defined as in ^W\j. 
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Proof. By the definition of g and p2]) . we see that g is differentiable at [a^ ^b'^Y' ^"^^ 
g' = K. Then the assertion follows from ([9]) and (|10p . □ 

The dependence of Kronecker product of matrices for K makes the computation of 
Kg(j4, 6) via (|14p too costly. The same are the formulas given in [SI [18]. For a computational 
purpose, we will present a new formula of Kg{A,b) that has a simpler and clearer form. 
To this end, we need a lemma. 



Lemma 1 Let 

Then C is positive definite. 
Proof. Noticing that 



C=AA + (T„+i/„ - uu2^-y 



C = A^A-al,Jn + 2al,Aln 



XX 



1 + X 



(15) 



and that A^ A — (yj^\-^In and — if jf^.||!j siiia both positive definite, we complete the proof 



of the lemma. 



□ 



Theorem 4 Let A^A + 

c^+i-^n — [[""nil-. — — LL^ be the Cholesky factorization. Then 



,{A, b) = vIRFTl II {A^A - al^^Ln)-^L\ 



(16) 



Proof. Consider expression ()13p of K. By the properties of Kronecker product of matrices, 
we get 



G{x)G^{x) = ([x^ -1]®/™) 



and 



Thus, we have 





X 


( 


-1 


X d 


3 Im 







L 



= (||x||2 + l)/„, 

(/„ (g) r'^){x (g) Lm) = xr'^ 



[Ln (S) O] 



Ln®r 



O 



{In®r'^){Ln®r) = \\rfln- 



T 

\r\\ \\r\ 



2A^-f-^G{x) - A^Gix) - [In ^ r'^O] 



In®r 
O 

2 , ^\ aT A , iui|2r ^JT a aT T 



2G^(x) — — ^-G^(x)^ 



\x\\^ + A + \\r\fln - xr^ A- A^ rx 
\xf + l)A^A + \\rfln - 2al,^xx'^ 
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The last equality used A^rx"^ = a^^^^^xx'^ , which follows from ([7]). Denote P = A^A 

KK^ = p-'^{{\\xf + l)A^A + \\rfln-2al^^xx^)p-^ 



crl^iln- We get 



= {\\xr + l)P~' ^A'A + ai^,In 
In the last equality we used ([U]). From Theorem [31 we have 



llxiP + 1 



p-\ 



(17) 
(18) 



KgiA,b) = i\\xf + l) 



□ 



Based on Lemma [TJ we complete the proof. 

Compared with the formula of Kg[A, h) in Theorem [3l the formula in Theorem S] does 
not involve the Kronecker product of matrices and makes its computation convenient. 
However, if (T„ and (T„+i are close, then A^ A — a'^j^^In becomes ill conditioned. Therefore, 
it may be hard to use (fTU]) to calculate Kg(A, h) accurately. Next we derive a new formula 
that can be used to compute the condition number accurately. 



Theorem 5 Let U^[A b]V = diag(cJi, . . . , fT„+i) be the SVD of [A b] with V = [vi, 
Denote Vn = V{1 : n,l : n). Then 

KgiA,b) = ^/\\xf + l \\VfJS\l 



, Vn+l\ 

(19) 



where S = diag([si, . . . ,s„]), Si = , i = 1, . . . ,n. 

Proof. Denote P = A^ A - a^^^In- From p^ . we have 



\xP + 1 



KK^ = P-^ + 2al^^p-' [In 



XX 



1 + X 



p- 



(20) 



Note that 



n+l n+1 

[A bf[A b] - al^Jn+i = ^ crfvivj - cr^+i ^ Vivf 

i=l i=l 
n 

We get 

n 

P = A^A-al_,Jn = [/nO]^(^72-a2+J 



i=l 



i=l 



[In 0] [f 1 , . . . , Vn] 





r T 1 




. n _ 



(21) 
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Similarly, by dS]), since Vn+i 



1 + X 



XX —X 
-X 1 



, we have 



In+l - Vn+lV.^j^i = [vi,...,Vn\ 



T 



and 



XX 



1 + ||X||2 

By (p2]) . we see that Vn is invertible. Combining ([2T]) and (p2]) . we have 



(22) 



'1 



T a-Ita-1 



T /tz-Ta-It/^-I 



(23) 
(24) 

Then by ()20p and Theorem [3] we get the desired equality. □ 

By Theorem [51 we can calculate Kg{A, b) by solving a linear system with the coefficient 
matrix V^. Next we show what the condition number of is exactly. 



Theorem 6 For Vn, we have 
and 



11) 



(25) 



(26) 



Proof. By the definition of Vn and the interlacing property [17, p. 103] for eigenvalues of 
symmetric matrices, we get 



ai{Vii) = l,...,an-i{Vii) = 1. 



Noticing that 



1 + X 



XX \ X = X 



1 + X 



1 + X 



rx, 



we know that is an eigenvalue of VnVii, that is, an{Vii) 

have proved ([25]) and ([26]) . 



= . Thus, we 



□ 



A different SVD-based closed formula for Kg{A,h) appears in [Ij. It is shown in [T] 



that 



,{A,h) = + 1 D [y^ On,l\ V [D On,l] 



(27) 
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where 



D 
D 



diag 
diag 



(T; 



n+l 



cr, 



n+l 



2 2 

n+l' • • • 1 V 



Compared with ([27|) . our (fT9]) is simpler and more compact. Furthermore, ([27|) depends on 
the singular values and right singular vectors of both A and [A b]. In contrast, (I19p involves 
only singular values and right singular vectors of [A h\. Therefore, the computational cost 
of the condition number by (jl9p is half of that by (j27p . Furthermore, the following example 
shows that the computed results by ([T9]) can be more accurate than those by ([27]) . 

A small example. We construct a TLS problem with (7„ and On+i very close. We 
generate A, b by [A b] = generateA6a(m, n, a) (see Appendix) by taking m = 15, n = 10, 
a = 10"^ 



0"n+l/0"n 








Kim 


0.608 


1 - 1.49 X 10^^^ 




1.13 X 10^ 


3.12 X 10« 



In the table, (t„+i/(T„ and (T„+i/ct„ denote the quotients of o"„+i over an and 
(Tn, respectively. Kg (fT4|l . Kg (fT9]) and Kg (|27|l denote the computed K'^g{A,b) by calculating 
Kg{A,b) via (fH|) . (fT9]) and ([27|) . respectively. (T„+i and (T„ being so close makes A^A — 
cr^+iln numerically singular and makes Kg (fH|) unreliable completely, so the result of K^ p^ 
is omitted. 

We comment that Kg (|19p is reliable as, by the remark in Appendix and Theorem [6l 
k{V^) = = 10^. This means that computing Kg{A,b) via (I19p amounts to solving 
a moderately ill-conditioned linear system. Furthermore, the right-hand side S of the 
system can be constructed with high accuracy since o"„+i and cjj are not close: < 
= 0.608, z = 1,2,... ,re. In contrast, Atg pTi) is inaccurate since computing Kg{A,b) 

via (I27p involves the diagonal matrix D and the closeness of fin+i (about 0.299) and an 
makes its last diagonal entry both very large (about 10^^) and very inaccurate in finite 
precision arithmetic. 

4 Straightforward bounds on the TLS condition number 

4.1 Sharp lower and upper bounds based on SVD of [^4 b] 

In this subsection, we further improve our result in Theorem [5] from the viewpoint of 
computational cost. We will show that with the SVD 

U^[A b]V = diag{ai,...,an+i) 

we are capable of estimating Kg(A,b) accurately based on the singular values of [A b] 
and the last row of V without calculating 



V{fS , where Vu = ^(1 : n,l : n), S 



I — — ^ 

diag([si, . . . , s„]). Si = a""*"^ , i = 1, . . . , n, as defined in Theorem^ 



^n+l 
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From now on we denote a = , , which is always smaher than one for x 7^ 0. 

Vi+ll^lr 

Keep (p5]) in mind and note that 

si < S2 < • • • < Sn- 

We then get 

Sn = crn{Vn^)\\S\\ < \\V^-/S\\ < ll^n'^IIIISII = a-^n- 
Therefore, from Theorem [5] we get 

K := a^^Sn < Kg{A, b) < R := a~^s„. (28) 

So, if a ~ 1, that is, Vn is nearly an orthogonal matrix, the lower and upper bounds in 
(f28]l must be tight. 

More generally, for a not small, say, ^ < a < 1, we have k < 4s„ and k > s„. So 
K<R< 4k. Therefore, in this case, our lower and upper bounds on the condition number 
Hg{A, b) are very tight and can estimate the condition number accurately by no more than 
four times. 

In the following, we only need to discuss the case that a < ^. It will appear that we 
can establish some lower bound k and upper bound R such that k < k < 4k still holds. As 
a result, together with the above, for any < a < 1, we can estimate Kg (A, b) accurately. 

Lemma 2 V can be written as 

\/l — y-^ —a ' 

where Un and Vn are the left and right singular vectors associated with the smallest singular 
value of Vii. 



Proof. Based on Theorem [6l we let 

Vu = U 



ln~l 



a 



be the SVD of Vu, where U = [ui, . . . , n„] G 



\V = [vi,...,Vn] G 



and U^U 



V V = In- It is easily justified from ([4]) that \V{n + l,n + 1)| = a. Without loss of 
generality, we assume V{n + l,n + 1) = —a. Then, by the theorem in Section 4 of jll] . 
we get 



V 



U 



U 



In-l 
Ol,ri-l Ot 



On-1,1 

—a 



Ln-l 



a 



a^Un 



a/1 — a^fj 



—a 



the desired form of V. 



□ 
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Following Lemma [2] and letting [/3i, . . . , —a] be the last row of V , we have 

^ :[/3l,...,/3n]. 



Noticing that (a ^, Un, Un) is the largest singular triplet of V^^-^ , we denote by 



11 



[Un.,Ui . . . ,n„_ij 



-1 



'^n-l 



which is the SVD of F^^'^. Then, by ([291) we have 



11 



/3i 



«-Mi 



^^n-l(l) 
3fc 



^)i(A:) 

Vn-l{k) 



a ^l3k 



Un + Wk, 



where Vi{k) denotes the kth component of Vi, Wk = Y17=i 'Vi{k)ui, k = 1, . . . ,n. 
Lemma 3 For given matrices Ai,A2 G M"^", if AJA2 = O, then 

l{\\A^\\ + \\A2\\)<\\A,+A2\\. (31) 

Proof. For an arbitrary vector x G M", from (Aix)'^ {A2X) = it follows that 

\\Aix\\,\\A2x\\ < \\Aix + A2x\\ 

and that 

||yli|| = max||^||=iPix|| < max||^ll=ipix + ^2x|| = Pi + A2II, 
P2II = max||^||=i||A22;|| < max||^,||=i||^ix + ^2x11 = 11^1 + ^2||- 

So, we get the desired inequality. □ 
To prove the main results of this section, we need the following two propositions. 

Proposition 1 Let . . . , — a] be the last row of V , Vu = V{1 in,! : n) and 
S = diag([si, . . . , where si,...,Sn are arbitrary positive numbers and satisfy < 
si < S2 < ■ ■ ■ < Sn- Then 



VT 



vi{n) 

-.Un + Wn 



(29) 



(30) 



< 



VTi^s 



+ 



VT 



< c 



VT^ 



+ Sn- 
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Proof. Following ([30|) . we get 



\/l — a' 



-.Un + SiWi, 



=Un + SfcWfc, . . . , , ^ Un + Sn.Wn 



Define 



A, 



a ^/3isi _ a ^/3„s^ 



Mr; 



Then V{{^S = Ai + A2. Noticing that 

u'^Wk = 0, k = 1, . . . ,n 

we get AJA2 = O. Thus, we have 

1 



, A2 = [siWi, . . . ,SnWn] 



:(Pi|| + P2||)< 



<Pl|| + P2||, 



(32) 



in which the left-hand side inequality follows from Lemma EJ Furthermore, noticing that 

-1 



a 



y/l — a 



and lliir, II = 1, we have 



Pi 



a 



In the meantime, note that 



||[/3lSi,...,/3„Sn] 



a 



(33) 



\Wr, 



\\[wi,...,Wn] 



[Ul, . . . ,Un-l] 



-T 

vi 



-T 



and 
From 
we get 



\\S\\ = -Sn. 
\SnWn\\ < P2II < ||[U'1, ...,Wn 



-Sn < \\A2\\ < Sn- 



Combining (j32|l . (j33l) and p4]) . we establish the desired inequality. 



(34) 
□ 
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Proposition 2 Suppose that a < ^. Then for c and c in PropositionUl we have 

C<C<4:C. 



Proof. If J^" ^ ^ < then it is easy to verify that 



Vl-a2-/32 ^ 1 



and 



c > -c. 

4 



Thus, (ESI) holds. If -jhL^ > ^, then 



Q : > ——a > 1, 



so a 



-1 



^■Sn > Sn, from which and the definitions of c and c it fohows that 



< 



Thus, dMl) still holds. 

Now we are in a position to derive sharp bounds on Kg (A, h). 

Theorem 7 Let . . . , /3„, —a] 6e t/ie last row of V and S = diag([si, . . . , 
i = l,...,n. T/ien 



1 a-^y/^lsi + ... + /3^sl , VI - - -1 



+ 



-Q 



< Hg{A,b) <R:-- 
Moreover, if a < ^, then 



_ +0; Sri. 



K,<K< 4k. 

Proof. Noticing that < si < S2 < • • • < Sn and using Proposition (TJ we have 

1 fa-^^P'fsi + ... + t3lsl ^ Vl - - /3n \ 



+ 



< 
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By Theorem [5l we get the first part of the theorem. Furthermore, we have the second 
part of the theorem by Proposition [2j □ 

A small example (Continued). From Theorem [71 we have 

5.65 X 10*^ < K'g{A,b) < 1.13 x 10^ 

The lower and upper bounds estimate Kg (fT9]) = 1.13 x 10^ accurately, as described in the 
second part of Theorem [71 



4.2 Lower and upper bounds based on a few of singular values of A and 

[Ab] 

In |10j . bounds on the condition number of the Tikhonov regularization solution are es- 
tablished based on a few singular values of A, where A is the coefficient matrix of the 
least squares problem under consideration. This is particularly useful for large scale TLS 
problems since for them any formulas and bounds for the condition number involving all 
the singular values of A and/or [A b] are too costly to be computed. Such a bound can be 
obtained by computing only a few singular values of A and/or [A b]. 

In the following theorem, we establish similar results for the condition number of the 
TLS problem. 



Theorem 8 We have 

Hi < fig{A,b) < Ri, (36) 

where 



Proof. Denoting 

M = {A^A - al^Jn)-^ {{\\xf + l)X'A + ||r||2/„) {X' A - al^Jn)-\ 
from (117p we have 

KK^ = M- lal+M^'A - al+JnY^xx^iA^A - al^Jn)'^ . (38) 

Here and hereafter, Aj(M) denotes the zth largest eigenvalue of M, where M is an arbitrary 
symmetric matrix. By the Courant-Fischer theorem [14^ p. 182], from (j38p we get 

\2{M) < \i{KK^). (39) 

Furthermore, since 2a'^j^^{A^ A — a'^_^]^In)^^xx'^ {A^ A — a'^j^]^In)~^ is nonnegative definite, 
the following inequality holds 

XiiKK"^) <Xi{M). (40) 



Collecting (j39j) and (j40|) and based on p4|) . we have 

^/X^) < Kg{A, b) < ^/MM). 
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It is easy to verify that the set 

[ + l)a| + ||rf ' 

1 



n 



consists of all the eigenvalues of M. We define the function 

+ l)a^ + ||rf 

and differentiate it to get 

-2a^{\\xf + 1) - 2cT(||xf + l)al^^ - Aa\\r\ 



It is seen that /'(cr) < and f{a) is decreasing in the interval (ct„+i,oo). Thus, we get 
that 

Ai(M) - — ' -^slM) - -2 -^-2 

and 

kP + i)'^Li + lkP /^^pTTWTHP 



^2 2 — '"y V-' '-^/ — ^2 2 

II -112 

Noticing that = ct^^;^, we complete the proof. □ 

Remark. In Corollary 1 of [T], the authors prove that 



K'giAb) < 2 



Since (t„ < (5"i, iTi < cii, we get 



Kl < TT^ o < 



Therefore, our Ri in (I37p is sharper than the above upper bound. 

It is seen that the lower and upper bounds on Kg{A,b) in Theorem [8] are marginally 
different provided that an and an-i are close. This means that in this case both bounds 
are very tight. For the case that (T„ and dn-i are not close, we next give a new lower 
bound that can be better than that in Theorem El 

Theorem 9 It holds that 

where Ri is defined as in Theorem and 



^/T+W\ 

^2 ~ 
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Moreover, when (5"„_i > ct„+i + y ct,^ — cr'^j^^, we have 

—\ — ^2 * 

Proof. Denote P = A^A - c7.^+i/„. From ([20]), we have 

T 



Noticing the second term in the right-hand side of the above relation is positive definite, 
we have 

+ l)Ai(p-i)<Ai(i^if^), 

that is, 

llxlP + 1 



<<{A,b). 



Thus, the first part of the theorem is obtained. 

The second part of the theorem is proved by noting 



O-n-l + <7^+l 1 



< < 



under the assumption that dn-i — > y o"^ — c^+i- n 

Remark 1. At first glance, the assumption in the second part of the theorem seems 
not so direct but we can justify that it indeed implies that an and ct^-i are not close. 
Actually, we can verify that the second part of Theorem [9] holds under a slightly stronger 
but much simpler condition that 

O'n-l > 2(T„. 



Remark 2. From 



^2 ■ - "L, 



1 + 



j2 



'n + 1 



it is seen that ^ > 1 provided cr„+i > 0. Only for cr„+i = 0, ki = K2 holds. At this time, 
h G TZ{A) and r = 0. 

We observe that the bounds on Kg{A,h) in Theorem [9] are tight when is small, 
compared with one. On the other hand, once -^^-^ is not small, these bounds may not be 
tight. For this case, we will present new bounds that may better estimate Kg{A,b). 

The proof of the following theorem depends strongly on Propositions [J and [21 
Theorem 10 Assume that a < i. Denote p = ^^i±i. Then 



^l + \\xf ^ , , - /1 + 3I/92 ^l + \\xf 
^ <Kg[A,h) < K2:=\l- ^ =. (41) 



2 2 y 1 — p /~2 2 
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Proof. Based on Theorem [9l it suffices to prove the right-hand side of (j4ip . Prom 
20]) and ([23]), we get 

T 



1 



\x¥ + 1 



KK' 



-T A -1t/-1 



XX 



1 + X 



where P = A^A - al^^In, A = diag([o-f - al^^, ■■■^crl- al^^]). Denote 



(42) 



D = diag([di,...,(i„]), 



T = diag([ti, . . . ,t„]), 



1 



2 ^2 



,i= l,...,n, 



,i = 1, . . . ,n. 



crj — cj; 



n.+l 



Then = (^Fn^^j (^^n^) and = 2 (y^^^ {DV^^^). 

Note that < di < ^2 < • • • < d„ and < ti < ^2 < • • • < ^n- Applying Proposition [H 
we get 



1 ( a- V/5i'rff + ••• + « ^ ^l-a2-/32 ^^^ 



< 



(43) 



and 



+ — Z-n 



< 



respectively, where [/3i, . . . , /3n, —a] denotes the last row of V as before. Define kn = 

dn-l 



, . Then 



Thus, by (P3|) we have 



0"n+l 



< A:. 



2 2 



a 



^pfti + ... + ^?M 



+ tn 



(44) 



Note that for the lower and upper bounds on 
that 



above, by Proposition [2] it holds 



a 



+ tn < 2 
< 4 



~T , tr, 



(45) 
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Based on (HH) and ([35]) . we derive that 



-^\\E\\^<4kn V~,'^T =4/c„||P"M|5 



^/2 

and that 

ll^ll < 32A:^||p-i||. (46) 
Combining (I46p and (|42p . we establish that 

= lli^ll = lli^i^^ll^ < Vl + 32A:2v^l + ||x||2||p-i||i 



'l + 31p2 y/1 + ||x||2 



2 



cr„ — a: 



n+l 



So, the proof of the theorem is completed. □ 

Remark. It is clear that the bounds in Theorem 1101 are tight when p = is small, 
compared with one. The result in this theorem is of particular importance in the case 
that is close to one. Recall that the lower and upper bounds in Theorem [9] differ 
considerably when is close to one. Theorem [TO] tells us that, if only ^^^^ is not so 
close to one, ng{A^ h) should be close to the lower bound. 

The improvement of R2 to Ri can be illustrated as follows. For small, i.e., (Jn+i 
and (T„ not close, as an upper bound of 6), 



■.= ^\\[Ab]\\F 



'l + 31p2 Vl + ||x||2 \\[Ab]\\F 



'1 + 31/^2 II [^5] 



. _ ^2 



is a moderate multiple of —j=====. In contrast, 



Vl + ||x||2 V^^^ + ^n+l 



< + <+l 



n-2 - n-2 



ll[^&]|| 



is a moderate multiple of .2 ^ — • The improvement of K2 over becomes significant 

as (Tn+i and (T„ are close. Similarly, R2 also improves the approximate condition number 
used in [2]: 

■ (Tn - CT„+1 a2 - a2_^i 

We will further illustrate the improvement by numerical experiments to be presented 
in Section 5. 
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5 Numerical experiments 



We present numerical experiments to illustrate how tight the bounds in Theorems [9] and 
[To] are, and to compare the bounds with the related result in |2j. For a given TLS problem, 
the TLS solution is computed by All experiments were run using Matlab 7.8.0 with 
the machine precision emach = 2.22 x 10~^^ under the Microsoft Windows XP operating 
system. 

Example 1. In this example, the TLS problem comes from [7J. Specifically, an 
m X (m — 2uj) convolution matrix T is constructed to have the first column 



V2 



Tva'^exp 



i = 1,2,... ,2w + 1, 
otherwise, 



and the first row 



t 







if j = l, 
otherwise, 



where a = 1.25 and w = 8. A Toeplitz matrix A and a right-hand side vector b are 
constructed as A = T + E and b = g + e, where ^ = [1, . . . , 1]"^, -E is a random Toeplitz 
matrix with the same structure as T and e is a random vector. The entries in E and e 
are generated randomly from a normal distribution with mean zero and variance one, and 
scaled so that 

||e|| = 7II5II, ll^ll = 7||f'||, 7 = 0.001. 



Table 1: 



m 


Cn+l/Cn 




4(^,6) 


H2 


K2 


— r 

k[ 




100 


0.981 


1 - 7.85 X 10"^ 


7.70 X 10^ 


7.04 X 10^ 


2.01 X 10^ 


7.94 X 10^^ 


1.03 X 10^^ 


300 


0.995 


1 - 2.05 X IQ-^ 


1.40 X 10*^ 


1.26 X 10* 


6.90 X 10^ 


8.83 X 10^^ 


6.54 X 10^'^ 


500 


0.998 


1 - 5.66 X 10-« 


9.01 X 10'' 


7.89 X 10'^ 


6.56 X 10^ 


3.32 X 10^^ 


1.90 X 10^" 



In the table, 

= f^\\[A b]\\F, = ^\\[A b]\\F, R\ = ^\\[A b]\\F, 
\\x\\ \\x\\ \\x\\ 

see Theorems 1101 and [9t respectively. We calculate the approximate condition number used 
in [2]: 

-r 

'^f2l = ~ • 

As indicated by the table, all the given TLS problems are similar in that (T„+i and (T„ 
are close but an+i and an are not so close. As estimates of 6), the lower bounds 

K2 are very accurate, and the upper bounds K2 improve the corresponding and R^ 
significantly by one or two orders. 
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Example 2. In this example, the TLS problems are generated by the function de- 
scribed in Appendix. For given m, n and a, A and h are generated by 

[A h] = generate746a(m, n, a). 

A different a gives rise to a different TLS problem with different properties. As a becomes 
smaller, an+i and ct„ become closer, so that the TLS problem becomes worse conditioned. 
For each of the TLS problems, we calculate the same quantities as those in Example 1 
and list them in Table 2 in which the first set of data is for (m, n) = (500, 350) and the 
second set is for {m,n) = (1000, 750). 



Table 2: 



a 








^2 


^2 


— V 

k{ 




10" 


2 


0.953 


1 


- 3.05 X 10" 


-4 


2.55 X lO'' 


8.98 X 10^ 


1.60 X 10^ 


5.14 X 10^ 


6.29 X 10^ 


10" 


3 


0.980 


1 


- 3.16 X 10" 


-6 


2.01 X 10^ 


8.75 X 10^ 


2.42 X 10^ 


4.92 X lO'^ 


6.03 X lO'^ 


10" 


-5 


0.953 


1 


- 2.77 X 10" 


10 


1.97 X lO'^ 


9.78 X 10^ 


1.74 X 10^ 


5.87 X 10" 


7.20 X 10" 


10- 


-7 


0.966 


1 


- 1.80 X 10" 


14 


3.28 X 10^ 


1.12 X 10^ 


2.38 X 10^° 


8.38 X 10^5 


1.02 X 10^6 


10" 


•2 


0.983 


1 


- 2.78 X 10" 


-4 


6.76 X lO'' 


1.65 X 10"* 


4.97 X lO*' 


9.90 X 10^ 


1.21 X 10** 


10" 


3 


0.978 


1 


- 1.95 X 10" 


-6 


6.70 X 10^ 


1.93 X 10^ 


5.09 X 10^ 


1.38 X 10*^ 


1.69 X 10® 


10" 


-5 


0.968 


1 


- 3.01 X 10" 


10 


4.33 X 10^ 


1.60 X lO'^ 


3.52 X 10® 


9.24 X 10" 


1.13 X 10^2 


10- 


7 


0.993 


1 


- 3.82 X 10" 


14 


1.13 X 10^° 


1.44 X 10^ 


7.02 X 10^° 


7.38 X 10^5 


9.03 X 10^^ 



We can see from the table that, for a = 10 ^ in which cj„ and Un+i are not very close, 
Ri and Kgj are very tight and they estimate Kg{A,b) quite accurately; for a < lO""'^, (T„ 
and an+i become closer with decreasing a, Ri and R^ estimate Kg{A, b) increasingly more 
poorly. In contrast, however, for all the cases, since an and an+i are not so close, K2 and 
R2 estimate K'^g{A,b) accurately. Particularly, for a < 10~^, R2 improves R^ and Kgj very 
considerably by several orders. 



6 Concluding Remarks 



In the paper, we have mainly studied the condition number of the TLS problem and its 
lower and upper bounds that can be numerically computed cheaply. For the TLS condition 
number, we have derived a new closed formula. For a computational purpose, we can use it 
to compute the condition number more accurately. We have derived a few bounds, which 
are quite sharp and can be calculated cheaply. We have confirmed our results numerically 
and demonstrated the tightness of our bounds by numerical experiments. 
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A Codes for generating tested TLS problems 



The following codes produce an mx (n+1) matrix [A b], which has the SVD [A b] = U'EV'^ 
with V{n + 1, n + 1) = —a, where < a < 1. 

[A b] = generateAba(m, n, a) 

% m,n : two given positive integers with m > n 

% a : a given positive number with < a < 1 

Generate V; % a random orthogonal matrix of order n 

V = generateV(n, V, a); 

B = rand(m, n + 1); % the Matlab function rand( ) 
[U, S, V] = svd{B, 0); % the Matlab function svd( ) 
[Ab] = U*T,*V'^ 

The subfunction generateV() is shown as follows. It is used to produce an (n + 1) x 
(n + 1) orthogonal matrix V with V{n + l,n + 1) = —a, where < a < 1. The idea of 
construction comes from Lemma O 

[V] = generateV(n, V, a) 

% n : a given positive integer 

% V : a given orthogonal matrix of order n 

% a : a given positive number with < q < 1 

partition V = [vi, . . . ,Vn]', 

generate U = [ui, . . . , % a random orthogonal matrix of order n 

Vii = [ui,... ,Un-l][vi, ■ ■ ■,V„-lf + aUnVn] 



V 



a^v^ —a 



Remark. Lemma 4.3 in [5j gives 

\ulh\ 



< 11x11 < 



2((5"n — CFn+l) O'n — 

Equivalently, it holds that 

TTT]— [7 < CTn - 0-j-t+l < 7]— n") (47) 

2||a;|| ||x|| 
where it is supposed that x 7^ 0. Note that Vin + 1, n + 1) = —a and a = , ^ From 
(j47p we see that a small a implies that a„ and an+i are close in some sense. 
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